1. Field of the Invention
The present invention relates to client-server based computer systems, and more particularly, to a method for benchmarking and optimizing the end-to-end processing performance of a client-server based computer system to determine the optimal values of the system variables.
2. Background of the Invention
Virtually every large business today relies on a computer network system to manage information. Typically, these systems operate multiple software applications to handle business functions such as accounting, purchasing, inventory, work scheduling, and customer service. The businesses invest huge sums of money to develop custom networks that efficiently meet these specific functions. In return for such large investments, the businesses demand the fastest and most reliable systems possible.
Conscious of these demands, system designers attempt to predict system demands, such as the number and frequency of transactions or the number of concurrent users. Based on these predictions, the designers assemble hardware components capable of meeting the anticipated demands. As apparent to one skilled in the art, the process is often little more than an initial educated guess followed by trial and error. Considering the large number of applications a system is typically asked to support, this unfocused approach is seemingly unavoidable because of the sheer number of hardware and software variables involved in assembling a system.
Faced with such a large number of variables, system designers must first focus on making the system work. Unfortunately, multiple components manufactured by different companies and the applications layered on top of them introduce a limitless number of possible programming problems. By trial and error, system designers adjust relationships between hardware and software until a workable model is found. Often, ensuring proper communication between components and accomplishing overall system operation are the designers"" only goals. Thus, designers are usually satisfied just to resolve programming problems and achieve system functionality, without even considering system efficiency.
If the system designers do set aside time and effort for system optimization, they typically focus on improving the performance of the individual system components. On the component level, designers can more easily understand and evaluate the variables affecting performance. In addition, the system designers can draw from the experiences and recommendations of individual component manufacturers. For instance, a hardware manufacturer, such as Hewlett Packard(trademark), knows in detail the factors that improve or degrade hardware performance. Database engineers understand completely the optimal conditions for efficient performance of a database, such as Oracle(trademark). Thus, typically designers optimize the performance of each of the components, incorporate the optimized components into the system, and expect that the overall system is operating at close to its most efficient capability. However, because these components depend heavily on each other and on the specific application of the system, this assumption is often incorrect. Plainly stated, optimized components do not always add up to an optimized system.
The strategy of component optimization is a relic of earlier monolithic processing environments. Monolithic refers to a network system designed, manufactured, and supported by a single manufacturer. In the early days of network computing, a single manufacturer supplied the network, was responsible for overall end-to-end processing performance and, as the designer of each component, had the knowledge of the individual components and the overall system to meet optimization needs. However, as the computer industry advanced and increased in complexity, manufacturers began to specialize and produce only a single component of a network. These manufacturers still furnished design and optimization support for customers, but only for their respective components. Thus, today, the customer, who is typically the resident network engineer of a business, is burdened with the responsibility of assembling optimized components that do not necessarily amount to an optimized network system.
The present invention is a method for evaluating and optimizing the performance of a client-server based computer network system. Specifically, the present invention provides a method for benchmarking the end-to-end processing performance of a computer network system comprised of individually manufactured components. The benchmarking method identifies performance-affecting system variables, measures their relative impacts on performance, and arrives at optimal settings for overall system optimization.
As most broadly defined, benchmarking is a way to measure the efficiency and effectiveness of computer system performance. Typically, benchmarking is used as part of wide scale information technology metrics, which is defined as the science of measuring the efficiency and effectiveness of computer hardware, software, and systems. Once a system is operational, an initial benchmark test is run in which the system performs the function for which it was designed. This benchmark test evaluates the performance of the system by some measurable unit, such as time or frequency of performance cycles. In addition, the benchmark test notes the respective values of the system variables.
As described above, these benchmarking tests are typically applied only to components of a computer network. As opposed to this component level strategy, the present invention assesses performance on a system-wide level, taking into account system component interrelationships. The present invention is primarily intended for actual computer network systems that are in use, and not necessarily to model systems. The present invention analyzes computer network systems devoted to particular applications, and identifies different system destabilizers for each system and each application. The present invention provides the following method of system performance benchmarking, which can be performed either manually, by system designers, or preferably automatically, by software executing the steps described herein.
First, a set of performance variables that impact system performance is identified. These performance variables, also referred to as system destabilizers, influence the performance of the software, hardware, and network processes. Examples of these variables include the number of central processing units (CPUs), the number of concurrent users, and the network protocols. Once the destabilizers are identified, a software matrix is constructed to represent and track the performance variables. Next, the functional (but not optimized) system is performance-tested. This initial performance test establishes the baseline for the system, which is recorded in the software matrix.
With a baseline established, the performance variables are pinned to constant values except for one variable, which is allowed to float through different values. While this one variable is floating, system performance is recorded in the software matrix, including metrics pertinent to the particular software application, such as processing speed, system access and congestion, and system response time. Readings for each value of the floating variable are recorded as the values change through successive thresholds.
Once a statistically complete sample has been taken, the floating variable is returned to its baseline value and the method is repeated with a new performance variable. A test sample is statistically complete when the test results demonstrate that a relationship between the floating variable and system performance does or does not exist, and demonstrates such a relationship to the degree of accuracy required by the specific application of the computer network system. After tracking the computer network""s response to changes in value of the different system destabilizers, the present invention presents a clear picture of which variables improve or degrade system operation and, in addition, how each variable influences the performance of other variables. The relative impact of each variable is shown in the software matrix. Educated with this knowledge, a system designer can then implement system changes that address the most influential variables and optimize performance. For example, if the software matrix shows that the volume of records an application must process is the primary cause of slow system performance, then a designer may simply install faster processing hardware to solve the problem.
In the first step of the method of the present invention, choosing variables that affect end-to-end processing performance depends largely on the specific architecture, components, and application of the system. Generally though, these performance variables fall under four categories: environment variables, database variables, application variables, and network variables. Environment variables concern the physical configuration of the system and its components, including for example, the number of CPUs, the amount of memory, data striping (how data is physically placed across a disk array), and port configurations. Database variables pertain to the configuration and operation of a database and include variables such as buffer size, sort area, and shared pools. Application variables depend on the specific application being executed, but generally include variables such as transaction mix, transaction rate, array insert size, and number of concurrent users. Finally, network variables concern the methods of data exchange between computer network components, and include, for example, protocols and bandwidths. The relative influence on the system of these performance variables depends on the particular assembly of components and the applications the system executes.
The present invention can be applied to any client-server based computer network. However, in the preferred embodiment, the present invention is applied to a client-server system with an application tier, a database tier, and a client tier. In hardware terms, this system is a personal computer, such as a computer with an Intel(trademark) processor, connected to a server, such as an Hewlett Packard HPT520, and a database, such as Oracle(trademark).
Accordingly, it is an object of the present invention to optimize the performance of client-server based computer network systems.
It is another object of the present invention to provide a benchmark for the end-to-end processing performance of a computer network system, to identify the component variables, the network variables, and the processing environment variables that most affect overall system performance, and to determine the optimal values of those variables that produces the most efficient overall network system possible.
It is another object of the present invention to provide a computer system designer with an understanding of the variables affecting the performance of a computer network system composed of individually manufactured hardware and software.
It is another object of the present invention to provide means for determining the capacity of a network application, such as the number of concurrent users a network application can support.
These and other objects of the present invention are described in greater detail in the detailed description of the invention, the appended drawings, and the attached claims.